Reduce finish_reason reliance#96
Open
Nick-Source wants to merge 1 commit intoSecretiveShell:masterfrom
Open
Conversation
Nick-Source
force-pushed
the
support-non-auto-tool_choice
branch
from
31d7e68 to
e2bc89f
Compare
Non-auto tool choices will result in a finish_reason of "stop" rather than "tool_calls". The current code assumes "stop" means there are no tool calls to be made when this isn't true. Instead, it should simply check for any available tool calls. Alongside, the last message in a chat completion stream isn't guaranteed to have the finish_reason. Instead we should check for any reasons to stop as we collect chunks and then use that at the end.
Nick-Source
force-pushed
the
support-non-auto-tool_choice
branch
from
e2bc89f to
1c88b33
Compare
Nick-Source
changed the title
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR addresses issues with how tool calls are detected in OpenAI chat completions by reducing reliance on the finish_reason field. The main issue is that non-auto tool choices return a finish_reason of "stop" even when tool calls are present, causing the current logic to incorrectly assume no tool calls exist.
- Replace finish_reason-based tool call detection with direct checking of tool_calls presence
- Track tool call state during streaming to handle finish_reason inconsistencies
- Simplify logic to only stop processing for "length" finish_reason or absence of tool calls
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
| streamChatCompletion.py | Replaces finish_reason logic with tool_call tracking and direct tool_calls checking |
| chatCompletion.py | Updates condition to check for tool_calls presence instead of relying on finish_reason |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| if fully_done or tool_call == False: | ||
| logger.debug("no tool calls found") | ||
| fully_done = True | ||
| continue |
There was a problem hiding this comment.
Use not tool_call instead of tool_call == False for boolean comparison. This is more pythonic and consistent with PEP 8 style guidelines.
Suggested change
| continue | |
| if fully_done or not tool_call: | |
| logger.debug("no tool calls found") | |
| fully_done = True | |
| continue |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Non-auto tool choices will result in a finish_reason of "stop" rather than "tool_calls". The current code assumes "stop" means there are no tool calls to be made when this isn't true. Instead, it should simply check for any available tool calls.
This is talked about by OpenAI staff in this thread: https://community.openai.com/t/new-api-feature-forcing-function-calling-via-tool-choice-required/731488/13
Interestingly, these recent tests show different results: https://community.openai.com/t/function-call-with-finish-reason-of-stop/437226/45
I couldn't find any conclusive reference of this on the Docs/API page, but regardless it should be safer this way to rely on the presence/absence of tool_calls.
Alongside, the last message in a chat completion stream isn't guaranteed to have the finish_reason. Instead we should check for any reasons to stop as we collect chunks and then use that at the end. The openAI python library follows a similar manner when processing chunks:
https://github.com/openai/openai-python/blob/e6c6757553bbdb777c31d0daf5916fb9e2b47ff8/src/openai/lib/streaming/chat/_completions.py#L361
Fixes #88